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AN  ENTROPY  MAXIMUM  PRINCIPLE  AND  INSTANTANEOUS  FAILURE  STATISTICS 


I.  Introduction 

In  this  paper  we  use  and  extend  some  concepts  in  information  theory  in 
the  context  of  the  statistical  description  of  device  failure.  In  particular, 
we  consider  here  failures  that  are  instantaneous  in  the  sense  that  a  failure 
does  not  depend  on  the  history  (e.g.  length  of  operation)  of  the  device  [l]. 
Failure  probabilities  are  characterized  by  an  order  parameter  that  corresponds 
to  the  time  of  failure  for  the  device  under  consideration.  The  corresponding 
Shannon  entropy  [2]  is  defined  over  the  full  range  of  failure  times.  We  term 
the  range,  the  epoch  for  failures.  We  consider  an  entropy  maximum  principle 
(EMP)  for  the  epoch  entropy  subject  only  to  minimal  constraints  necessary  to 
characterize  failure  statistics;  namely,  normalization  of  the  probability 
density  integral,  and  the  existence  of  a  mean  time  to  failure. 

The  resulting  probability  density  that  is  consistent  with  the  EMP  subject 

to  minimal  constraints  is  taken  to  be  fundamental.  However,  it  is  observed 

that  the  time  scale  on  which  the  failure  statistics  are  recorded  does  not 

necessarily  coincide  with  the  time  scale  on  which  the  probability  density 

ta^s  its  fundamental  form.  Time  scale  transformations  are  considered.  The 

maximum  entropy  density  is  considered  to  have  the  Shannon  form  independent  of 

the  time  scale  used  in  its  description.  The  resulting  epoch  entropy  is  itself 

taken  to  be  invariant.  This  principle  of  invariance  of  maximum  epoch  entropy 

provides  a  relationship  between  time  units  on  different  time  scales  that  is 

useful  e.g.  in  the  consideration  of  the  effect  of  size  or  mass  scaling  in 

particular  devices,  and  in  the  discussion  of  the  behavior  of  parameters  that 

arise  in  accelerated  testing.  Given  this  framework  and  motivation,  we  now 

turn  to  a  more  detailed  description  of  our  procedure. 

Manuacript  approved  April  30,  19J4. 
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II.  Entropy  Maximum  Principle,  Time  Scale  Transformations  and 
an  Invariance  Relation 

In  failure  statistics,  the  statistical  probability  of  failure  of  a  device 
is  described  in  terms  of  an  order  parameter  6=8-8o  where  8q  is  the  value  of  6 
when  failures  are  first  possible.  For  the  definition  of  Shannon  entropy  we 
need  a  dimensionless  probability  density  so  we  introduce  a  constant  unit  Tg 
for  the  order  parameter.  Then  if  f(8)d8/tg  is  the  probability  of  the  occur¬ 
rence  of  a  failure  between  8  and  8+d8,  it  follows  that 

J^f (8)d8/tQ  =  1  (1) 

where  f(8)  is  a  dimensionless  probability  density.  To  make  sense  as  a  failure 
probability  density,  f(8)  must  have  a  first  moment  that  corresponds  to  a  mean 
time  to  failure 

<8>  =  J^8f(8)d8/T0  (2) 

(1)  and  (2)  represent  the  minimum  set  of  conditions  that  arise  in  the  descrip¬ 
tion  of  instantaneous  failure  statistics. 

We  now  define  a  dimensionless  epoch  Shannon  entropy  in  terms  of  the 
failure  probability  density  as  follows. 

Se  =  -£f(eUnf(8)de/T0  (3) 

It  should  be  noted  that  Sg  is  an  implicit  function  of  8q.  We  now  require  that 
Sg  must  be  maximal.  In  particular,  in  the  absence  of  knowledge  about  other 
constraints,  we  maximize  Sg  subject  only  to  the  constraints  represented  by 
(1)  and  (2).  Introducing  two  Lagrange  multipliers  A^  and  in  the  usual 
way,  we  obtain  from  the  variation  of  Sg  that  the  maximum  entropy  occurs  for 

fM(6)  =  exp[-(A1+A26)]  (4) 
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Consistency  with  (1)  and  (2)  allows  us  to  determine  A^  and 


expOAp  =  T0/<0>,  A2  =  <0>_1 


Then  the  dimensionless  probability  density  for  maximum  entropy,  f '  (9) ,  has  the 


fM(0)  =  [t0/ <0>]exp[- (6/ <0>) ]  (6) 

The  maximum  epoch  entropy  can  itself  be  determined  to  be 

Sg  =  1  +  £n<0>  -  jentg  (7) 

There  are  several  important  observations  that  should  be  made  concerning 
(6)  and  (7).  First  from  (6)  we  see  that  it  is  the  exponential  distribution 
that  provides  the  maximum  epoch  entropy  subject  to  the  minimum  constraints 
necessary  to  characterize  instantaneous  failure  statistics.  In  this  sense, 
the  exponential  distribution  represents  the  fundamental  distribution  for 

M 

instantaneous  failure  statistics.  Second,  although  f  (0)d0/tg  is  independent 
of  the  choice  of  time  unit,  f  (0)  alone  takes  on  a  particularly  simple  form 
when  tg  is  taken  equal  to  <0>,  i.e.  it  is  equal  to  unity  at  0=0,  and  homo¬ 
geneous  in  0/<0>.  Thus  <0>  is  a  natural  time  unit  for  the  0  time  scale.  A 

M 

similar  simplification  occurs  for  Sg  which  just  becomes  unity  when  Tg=<0>. 

M 

However  for  any  choice  of  T0,  Sg  is  an  epoch  integral  whose  value  should  be 
independent  of  the  time  scale  on  which  it  is  described.  This  condition  of 
invariance  serves  to  place  a  stringent  condition  on  the  relationship  between 
Tg  and  the  unit  of  time  for  the  new  time  scale.  This  is  probably  obvious  for 
a  linear  transformation  but  we  are  interested  here  in  a  wider  class  of  time 
scale  transformations.  We  are  motivated  here  by  the  fact  that  there  is  no 
reason  to  believe  that  the  time  scale  on  which  failures  are  recorded  coin¬ 
cides  with  the  time  scale  on  which  the  failure  statistics  distribution  appears 
in  its  fundamental  exponential  form. 
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To  provide  a  contextual  framework,  we  now  make  some  background  observa¬ 
tions  about  time  scales.  A  time  or  order  parameter  is  just  a  positive  cumula¬ 
tive  function  that  increases  monotonically  from  an  origin.  We  have  already 

^  <y 

introduced  the  8  time  scale,  its  corresponding  difference  time  scale  0=0-0o, 

and  unit  of  time  tg.  In  a  similar  way  we  introduce  another  time  scale  £  with 

difference  time  scale  V  ,  and  unit  of  time  t...  From  the  context  of 

o  t 

application,  it  is  clear  that  we  should  align  the  respective  origins  0Q  and  £q 
since  both  represent  the  initial  time  when  failures  can  occur.  Thus  if 

0=0(t) ,  0(t=O)  =  0  (8) 

We  specify  further  that  d0/dt  is  positive  and  finite  everywhere  except  pos¬ 
sibly  at  isolated  points. 

We  can  now  describe  the  failure  statistics  and  maximum  entropy  in  terms 
of  t.  Namely  by  direct  transformation  from  r  (0)d0/tg  to  gu(t)dt/tt,  we  find, 
using  (6) 

gM(t)  =  exp{-[0(t)/<0>] }  [lt/<0>][d0(t)/dt]  (9) 

The  maximum  epoch  entropy  can  now  be  written  in  terms  of  an  entropy  density 

M 

defined  in  terms  of  g  (t),  and  an  integral  over  time  on  the  t-scale. 

St  =  i^8MCt)AngM(t)  dt/Tt  =  1  +  £n<0> 

-  -Cexp[-(  4> )]  £n(Tt  a? }  (10) 

Within  the  integral  on  the  right  hand  side,  d0/dt  is  expressed  as  a  function 
<0>  and  0/<8>.  We  now  use  the  requirement  that  S  is  invariant  when  expressed 
in  terms  of  a  transformed  time  scale,  and  (7)  and  (10)  to  obtain  a  general 
invariance  condition. 

£n(TQ/Tt)  =  J^exp[-(8/<0>) ]£n[d0/dt)d8/<0>  (11) 

We  can  investigate  the  ramifications  of  this  relationship  by  considering  a 
specific  time  scale  transformation. 
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III.  An  Important  Example  and  Parameter  Renormalization 

Given  this  general  formulation,  we  can  now  turn  to  an  important  example 
of  time  transformation  that  will  provide  a  Weibull  failure  probability  den¬ 
sity.  Namely,  we  consider 

0(t)  =  atP,  de/dt  =  aptP_1  ,  a,p>0  (12) 

so  that 

dO/dt  =  ap(6/a)(P'1)/P  (13) 

The  result  of  using  Eq.  (13)  in  Eq.  (11)  is 

m  r  e<s>  f<0>/all/p*  =  _(1  -  jb*  (14) 

where  y=0-577215.  .  .  is  the  Euler  constant.  If  we  now  choose  Tq  to  coincide 
with  the  "natural"  unit  on  the  0-scale,  namely  <0>,  we  obtain  the  relationship 

<0>  =  otfJ^exp [  (I-P)yJt^  (15) 

It  is  convenient  to  define  a  new  unit  xt, 

Tt  =  pexp[  (l-p)y/p]Tt  (16) 

so  that 

<0>  =  or  ttP  (17) 

M 

Then  using  Eq.  (9),  we  obtain  for  g  (t)dt/tt, 

gM(t)dt/tt  =  exp[-(t/tt)h  P(t/Tt)^_1  dt/tt 

=  gM(t)  dt/tt  (18) 

-N 

The  function  g  (t)/tt  is  just  the  usual  3-parameter  Weibull  probability  density 
(recall  E  is  implicit)  that  is  so  prevalent  in  the  empirical  description  of 


the  probability  of  instantaneous  failure  of  a  wide  range  of  devices.  The 
scale  parameter  is  just  the  unit  of  time  Tt,  and  the  shape  parameter  is  {$. 

We  are  oftentimes  presented  with  a  Weibull  distribution  for  instantaneous 
failures  rather  than  an  exponential  distribution.  The  point  we  have  been 
making  is  that  such  a  Weibull  distribution  occurs  on  a  time  scale  of  measure¬ 
ments  which  is  not  the  fundamental  time  scale  appropriate  for  the  device  of 
interest  as  indicated  by  the  EMP  subject  to  minimal  constraints.  Thus  param¬ 
eter  dependencies  that  enter  into  the  unit  of  time  on  the  measurement  time 
scale  are  therefore  only  apparent.  The  actual  dependencies  are  those  that 
enter  into  the  unit  of  the  fundamental  time  scale.  The  two  dependencies  are 
related  by  means  of  the  invariance  relation  obtained  from  the  specification 
that  the  maximum  epoch  entropy  is  invariant  under  a  time  scale  transformation. 

For  example,  consider  a  plastic  supporting  component  with  strength 
dependent  on  the  molecular  weight,  M,  of  its  chemical  building  blocks.  On  the 
time  scale  of  measurement,  we  assume  the  failure  statistics  to  be  Weibull.  On 
the  fundamental  scale,  we  take  the  mean  time  to  failure,  for  specificity,  to 
have  a  simple  monomial  dependence  on  M. 

<0>  =  kMb  (19) 
Then  the  Weibull  scale  unit,  Tt,  obeys 

T  a  Mb/P  (20) 

and  therefore  has  a  different  power  dependence  controlled  by  the  shape  param¬ 
eter  p.  If  now  we  consider  plastic  components  made  with  plastics  with  dif¬ 
ferent  values  of  M,  they  will  typically  have  Weibull  failure  statistics  with 
different  values  of  the  shape  parameter.  Our  prediction  predicated  on  the 
invariance  of  maximum  epoch  entropy  when  expressed  in  terms  of  the  Weibull 
distribution  is  that  the  M-dependence  of  i  will  vary  in  such  a  way  that  b  is 
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the  same  for  the  components  with  different  M.  Similarly  it  may  be  possible  to 
change  the  Weibull  shape  parameter  by  changing  the  conditions  of  operation 
e.g.  by  changing  the  environment.  Then  the  prediction  is  that  the  power  of  M 
that  enters  into  changes  with  the  shape  parameter  in  such  a  way  that  b 
again  remains  invariant. 

A  more  general  application  occurs  in  the  case  of  accelerated  testing.  We 
consider  then  the  device  of  interest  to  be  subjected  to  a  constant  stress 
which,  for  specificity,  we  take  to  be  the  absolute  temperature  T.  For 

simplicity,  we  assume  that  the  origin  of  time  for  failures  is  not  modified  by 
a  change  of  temperature.  We  now  assume  that  the  temperature  dependence  of  tb 
unit  of  time  is  described  by  an  Arrhenius  model  so  that  on  the  0-scale, 

<0>  =  <0^  exp(A/T)  .  I  , 

Here  <0^  is  the  mean  time  to  failure  at  (nominally)  infinite  temperature  and 
A  is  an  Arrhenius  constant  with  dimensions  of  temperature.  We  again  take  the 
distribution  on  the  time  scale  of  measurement  to  be  Weibull.  Then  if  under 
temperature  stress,  we  find  that  the  statistics  are  still  Weibull  with 
unchanged  shape  parameter,  we  can  apply  the  Arrhenius  model  to  the  Weibull 
time  unit. 

tt  =  (i^exp^/T)  (22) 

Here  A*  is  an  effective  Arrhenius  constant.  It  is  clear  that  for  the  invari¬ 
ance  condition  to  hold,  A*  must  be  related  to  the  actual  Arrhenius  constant  A 
by  the  renormalization  relation 

A*  =  A/p  (23) 

Despite  its  apparent  simplicity,  (23)  provides  an  important  insight  into 
the  validity  of  stress  testing.  If  the  shape  parameter  (3  is  less  than  unity, 
the  apparent  A*  is  larger  than  A  and  one  is  led  to  believe  the  device  is  less 
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durable  than  it  actually  is.  The  bad  news  is  that  if  |3>1,  the  device  is 
actually  less  durable  than  might  be  judged  from  the  stress  test  on  the  time 
scale  of  measurement.  The  important  point  we  want  to  make  is  that  conclusions 
concerning  accelerated  life  tests  depend  in  an  essential  way  on  the  details  of 
the  probability  density,  and  not  just  on  the  properties  of  the  measured  mean 
time  to  failure. 

In  summary,  we  have  formulated  for  instantaneous  failure  statistics,  an 
EMP  for  the  entropy  over  the  full  epoch  of  failures  subject  to  the  minimal 
constraints  that  the  probability  is  normalized  and  that  there  exists  a  mean 
time  to  failure.  This  EMP  led  to  an  exponential  distribution  for  the  proba¬ 
bility  density.  The  exponential  distribution  thus  can  be  viewed  as  the  funda¬ 
mental  distribution  for  instantaneous  failure  statistics.  We  then  considered 
the  family  of  probability  densities  that  can  be  obtained  by  time  scale  trans¬ 
formations.  The  condition  that  the  maximum  entropy  over  the  epoch  remains 
invariant  when  evaluated  in  terms  of  a  transformed  probability  density  led  to 
a  relation  between  a  unit  of  time  on  the  transformed  time  scale  and  the  one  on 
the  original  scale.  The  example  involving  a  time  scale  transformation  that 
leads  to  a  Weibull  probability  density  was  discussed.  In  particular,  it  was 
pointed  out  that  the  invariance  condition  imposes  important  constraints  on 
parameters  and  Arrhenius  constants  from  accelerated  testing,  that  may  occur  in 
the  description  of  the  time  scale  units.  Other  time  scale  transformations  can 
be  treated  in  a  similar  way  [3].  We  intend  to  address  the  application  of  an 
EMP,  time  scale  transformations,  and  the  invariance  of  maximum  epoch  entropy 
to  the  case  of  cumulative  damage  failure  statistics  at  a  later  time. 
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